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RISK  AND  THE  RIGHT  MODEL 


All  aspects  of  life  in  the  world  are  subject  to 
risk  and  uncertainty>  Risk  and  uncertainty  key 

char ac te r is t ics  of  any  long  range  planning  and  cost 
estimation.  Few,  if  any,  decisions  are  made  under 
conditions  of  certainty  and  without  risk.  Due  to  the 
complexities  involved,  analysts  and  decision  makers 
must  specifically  and  explicitly  address  this  risk  and 
uncertainty  in  performing  their  assigned  tasks.  Al¬ 
though  the  terms  risk  and  uncertainty  are  often  used 
interchangeably,  they  are  not  the  same.  Risk  i3  the 
probability  that  a  planned  event  will  not  be  attained 
within  constraints  (cost,  schedule,  performance)  by 
following  a  specified  course  of  act  ion  ,  [7 : 18 ] .  Uncer¬ 
tainty  is  incomplete  knowledge  [7:18].  1  Fisher  [3:202] 
says,  11 A  risky  situation  is  one  in  which  the  outcome  is 
subject  to  an  uncontrollable  random  ^event  stemming  from 
a  known  probability  dlstrifcatXerr;'  An  uncertain  situa¬ 
tion,  on  the  ot\x&±-~hrenTG~,  is  character ized  by  the  fact 
that  thjs_4H«Ttrsr&il  ity  distribution  of  the  uncontrollable 
r andgm'^event  is  unknown."  Canada  [1  :  252]  relaxes  these 
definitions  somewhat  by  concluding  that  risk  is  the 
dispersion  of  the  probability  distribution  of  the  ele¬ 
ment  under  consideration  while  uncertainty  is  a  lack  of 
confidence  that  the  probability  distribution  is  cor¬ 
rect.  "^lt  is  the  task  of  analysts  to  try  to  reduce 
uncertainty  to  risk  and  then  to  meaningfully  convey  the 
risk  to  decision  makers.  _ 

RISK  VERSUS  SEESITIVITT  AMALYSI8 

Analysts  cannot  eliminate  risk  and  uncertainty 
from  an  analysis.  At  best,  they  can  only  present  and 
explain  those  aspects  of  risk  and  uncertainty 
affecting  the  decisions.  This  is  done  through  risk  and 
sensitivity  analysis. 
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Rick  analysis  -ier  a  procedure  for  analysing  how 
randoanesa  affects  the  total  cost.  £-Ioj>lace  a  cost 
estimate  in  proper  perspective,  it  must  be  vTW*7^~ar  « 
random  variable.  By  definition,  a  random  variable  is  a 
numerically  valued  function  defined  over  the  sample 
space  (St  327].  Unfortunately,  the  application  of  risk 
analysis  seeas  Halted.  Authors  and  analysts,  such  as 
Large  (6),  McNlchols  (9],  and  Worm  ill]  have,  however, 
addressed  the  problea  of  risk  in  hardware  cost 
estimation. 


Uncertainty  is  addressed  through  the  application 
of  sensitivity  analysis.  Although  often  mistakenly 
used  as  a  substitute  for  risk  analysis,  sensitivity 
analysis  is  designed  to  systematical ly  explore  the 
implicatione  of  varying  assumptions  about  the  future 
environment  and  is  normally  centered  on  the  cost  driv¬ 
ers  where  a  range  of  alternative  parameters  is  investi¬ 
gated.  The  objective  is  to  identify  those  parasMters 
whose  change  will  impact  the  decision  at  hand.  Risk 
analysis  and  sensitivity  analysis  are  complementary 
and,  as  such,  are  a  vital  and  necessary  part  of  every 
cost  analysis. 

Iflnr  PERFORM  RISE  ABA  LYSIS 


Normally,  decision  makers  are  presented  with  only 
a  point  or  'most  likely*  cost  estimate,  with  no  indica¬ 
tion  as  to  the  risk  (variability)  in  that  astimate. 

For  example,  Figure  1  shows  the  relative  cost  of  two 
systems,  A  and  B.  Using  cost  as  tht  evaluation  crita- 
rion,  and  with  all  othar  factors  being  equal,  decision 
makers  would  choose  System  A  as  it  offers  the  lower 
cost.  But,  point  estimates  can  be  misleading  end  can 
lead  to  a  worse  decision  than  had  no  estimate  at  all 
been  uaed. 


System  A  System  B 
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Figure  1  Cost  Decision  Making 
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To  cite  an  example  from  Dienenann  [2:2-4],  Figure 
2  shows  four  cases  in  which  estimates  are  expressed  as 
probability  distribution?  to  reflect  the  actual,  though 
perhaps,  unmeasurable,  risk  surrounding  each  estimate. 
In  Case  I,  as  in  Figure  1,  decision  makers  are  faced 
with  no  real  decision  problem  because  all  possible 
costs  of  System  A  are  lower  than  System  Bj  using  the 
point  estimate  would  not  affect  the  decision.  The 
situation  in  Case  II  is  slightly  different  in  that 
there  is  some  probability  that  the  actual  cost  of 
System  A  will  be  higher  than  System  B.  If  this  proba¬ 
bility  is  not  large,  the  decision  makers  would  still 
select  A.  However,  when  the  overlap  is  significant, 
the  point  estimate  would  no  longer  provide  a  valid 
datum  for  system  selection.  In  the  third  case,  both 
point  estimates  are  the  same,  but  the  cost  distribution 
for  B  has  a  larger  range  or  variance.  Here,  decision 
makers  preference  toward  risk  must  enter  the  decision 
process.  If  they  prefer  to  minimize  risk,  they  will 
select  A.  Case  IV  is  a  more  complicated  situation 
where  the  expected  cost  of  System  B  is  lower,  but  much 
less  certain  than  A.  In  this  case,  if  decision  makers 
were  to  use  only  a  point  estiate,  they  could  easily 
make  a  wrong  or  undesirable  decision.  The  application 
of  risk  analysis  would  give  much  needed  visibility  into 
such  a  decision  problem. 

Unfortunately,  the  application  and  presentation  of 
risk  analysis  has  met  with  mixed  feelings.  Many  deci¬ 
sion  makers  were  only  interested  in  a  point  estimate 
[8:42-44].  There  are  four  predominate  reasons  for 
this.  First,  presenting  more  than  a  point  estimate  may 
constitute  an  information  overload.  Cost  is  but  one 
input  to  the  decision  process.  Information  presented 
must  be  clear,  concise,  and  easily  understood.  This 
leads  to  the  second  reason;  some  decision  makers  would 
not  understand  risk  analysis  and  its  associated  impli¬ 
cations.  Third,  the  possibility  of  high  costa  would 
cause  undue  concern  and  adversely  affect  the  decision. 
Fourth,  risk  analysis  would  impact  the  credibility  of 
the  study  giving  the  impression  that  analysts  were 
unwilling  to  stand  behind  their  analyses.  Most  deci¬ 
sion  makers  do  agree,  however,  that  analysts  should  do 
risk  analysis  for  their  own  benefit  and  in  support  of 
the  point  estimate. 
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But,  risk  analysis  provides  preciaely  the 
information  that  the  decision  makers  need.  If  alterna¬ 
tives  cannot  be  clearly  separated  and  evaluated  on  the 
basts  of  cost,  if  competing  cost  estimates  fall  within 
the  error  of  the  estimate,  then  the  decision  should  be 
based  on  some  criterion  other  than  cost.  If  the  proba¬ 
ble  cost  range  is  too  broad,  steps  should  be  taken  to 
refine  the  estimate  and  decrease  the  range.  Such  steps 
include  better  data  collection  and  improved  estimating 
methods  and  techniques.  If  the  possibility  of  high 
costs  is  so  significant  as  to  make  the  system  poten¬ 
tially  unaffordable,  decision  makers  should  be  aware  of 
this  prior  to  the  decision.  Ignoring  such  information 
does  not  lead  to  better  decision  making.  On  the  con¬ 
trary,  it  leads  to  cost  overruns,  unsuppor table  sys¬ 
tems,  and  impaired  readiness. 

MODELS  AND  MODELING 


The  models  or  mathematical  expressions  used  in 
cost  analysis  appear  in  two  general  forms:  the  addi¬ 
tive  model  and  the  multiplicative  model.  The  additive 
model  is  expressed  as 

Y  «  X,  +  +. . .♦  Xn  (1) 

i  l  n 

and  the  multiplicative  model  as 


where  X.  through  X  are  random  variables.  The  mathe- 
aaticalexpressions  used  in  practice  appear  to  be  more 
complex,  but  are  usually  reducible  to  these  two  general 
forms . 


Prom  earliest  grade  school,  it  is  taught  that 
multiplication  is  a  shortcut  to  addition,  and,  although 
this  is  true,  this  practice  can  have  unexpected 
consequences  when  doing  a  risk  analysis.  Substituting 
multiplication  when  addition  is  a  more  appropriate 
operation  will  cause  distortion  and  inaccuracy.  Thus, 
the  analyst's  choice  of  a  basic  model  is  very 
important.  As  an  example,  consider  the  two  alternative 
cost  estimating  relationships. 

Y  «  4X 

and 

Y'  -X+X+X+X 


(3) 

(4) 
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Although  they  may  appear  to  be  the  sane  at  first 
glance,  they  are  not.  The  expected  values  of  equations 
3  and  4  respectively  are 

E[Y]  -  4E  [X]  (5) 

and 

E  [  Y  '  ]  -  E[X]  +E(X]+E(X]+E[X]  -  4E[X]  (6) 

which  reinforces  the  notion  that  the  models  are  the 
same,  but  the  variances  of  equations  3  and  4  respec¬ 
tively  are 

Var  [Y j  -  42Var  [X]  -  16Var  [X]  (7) 

and 

Var  [Y ' ]  «  Var [X] +Var [X] +Var [X] +Var [X] 

-  4Var [X]  (8) 

Upon  examination  of  the  variances,  it  is  clear 
that  they  are  not  the  same  model.  The  point  of  this 
demonstration  is  to  reinforce  the  principle  that  models 
should  be  a  reflection  of  reality.  If  the  reality  of 
the  situation  suggests  aquation  3,  then  use  it.  But, 
if  reality  suggests  equation  4,  do  not  use  equation  3 
as  a  shortcut  for  representing  an  equation  4  situation. 

Next  consider  the  more  complicated  situation 
represented  in  the  multiplicative  case  by  the  product 
of  two  random  variables  and  in  the  additive  case  where 
the  number  of  terms  in  the  sum  is  itself  a  random 
variable.  The  multiplicative  model  is 

Y  -  NX  (9) 

and  the  additive  model  is 

Y'  *  X1  *  X2  XN  <10> 

where  X  and  N  are  independent  random  variables.  The 
expected  values  of  equations  (9)  and  (10)  are  respec¬ 
tively  ? 


E(Y]  -  E(N]  E [X]  (11) 

and 

E  ( Y  '  ]  -  E [N]  E  [X]  (12) 
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which  once  again  reinforces  the  notion  that  the  models 
are  the  sane.  The  variances  are,  however,  once  again  a 
different  story.  The  variances  of  equations  (9)  and 
(10)  are  respectively 

Va r  [ ¥ ]  -  E  [N] 2Var [X] +E (X) 2Var [ N ] +Var (N] Var [X]  (13) 

and  ^ 

Var(Y')  -  E (N)Var (X) +  £  [X]  Var [N]  (14) 

Once  again  there  is  a  significant  difference  in  the 
variances.  This  difference,  as  manifested  in  the 
multiplicative  model,  can  be  quite  misleading  to  the 
decision  maker  causing  risk  analysis  to  be  rejected  due 
to  absurdly  high  variances. 

As  a  practical  example,  consider  the  cost  of 
maintenance  personnel  for  a  theoretical  new  fighter, 
the  XF-1.  Let  X  be  the  pay  of  an  individual  mainte¬ 
nance  person  and  N  the  number  of  personnel  necessary  to 
support  a  squadron  of  the  new  fighter.  Expected  value 
and  variance  of  X  and  N  are  as  shown  in  Table  1. 


TABLE  1 


PX-1  Maintenance  Personnel 


X  N 

E  17,513  1763 

Var  23,703,700  35 
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Letting  HP  represent  total  maintenance  personnel 


cost , 


E [HP ]  -  1763  x  17513  -  30,875,419 

for  both  the  multiplicative  and  additive  models.  The 
variance  for  the  multiplicative  model  is 

Var [HP]  ■  1 76 3 2  x  23,783,700  +  175132  x  35  + 

23,703,708  x  35 

-  7.3687  x  1013 

By  the  same  token,  the  variance  of  the  additive  model 
is 

Var [HP]  -  1763  x  23,703,700  +  17,5132  x  35 

-  5.2524  x  1010 

which  is  a  reduction  of  three  in  magnitude.  This 
difference  is  significant.  Tolerance  limits  with  a  .95 
probability  level  that  95  percent  of  the  population  is 
included  in  the  limits  substantiate  this  conclusion 
[8:110-111],  The  limits  for  the  multiplicative  model 
are 


(14,050,595  ,  47,700,243) 
and  for  the  additive  model  are 
(  30,426,223  ,  31,324,615)  . 

Note  the  difference  in  the  range  covered  by  these  two 
intervals . 
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COM  C  LOS  I  OH 


In  many  cases,  the  additive  model  is  the  more 
appropriate  model.  If  individual  entities  within  the 
population  under  consideration  are  exactly  the  same, 
then  the  multiplicative  model  should  be  used.  But  if 
small  differences  exi3t,  then  the  additive  model  is 
more  justified.  To  illustrate  this  point,  consider  the 
cost  of  training  pilots.  If  the  cost  for  each  pilot  is 
exactly  the  same,  one  would  use  the  multiplicative 
model  and  arrive  at  total  cost  by  multiplying  the 
number  of  pilots  times  the  cost  per  pilot.  If, 
however,  the  cost  varies  among  individual  pilots,  the 
additive  model  is  more  valid.  These  cost  differences 
may  occur  if  a  particular  pilot  has  to  abort  a  flight 
due  to  aircraft  system  failure  and  then  reaccompl ish 
that  flight  or  if  a  pilot  needs  additional  flights  in 
order  to  attain  a  required  level  of  proficiency. 

Use  of  the  additive  model  is  further  reinforced  by 
most  data  collection  systems.  Data  are  normally 
gathered  and  aggregated  additively.  Good  examples  are 
accounting  systems. 

Risk  analysis  is  but  one  more  tool  in  the  hands  of 
the  decision  maker.  Unfortunately,  all  too  often  this 
tool  is  rejected.  One  reason  is  the  choice  of  models 
used  by  the  analyst  preparing  the  estimate.  This 
choice  must  be  a  conscious  one  for  the  ramifications 
are  great. 
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